Fringe Projection Profilometry Research Articles

Multi-view fringe projection profilometry for surfaces with intricate structures and high dynamic range

Multi-view fringe projection profilometry for surfaces with intricate structures and high dynamic range

Technology Selection for Inline Topography Measurement with Rover-Borne Laser Spectrometers

This work studies enhancing the capabilities of compact laser spectroscopes integrated into space-exploration rovers by adding 3D topography measurement techniques. Laser spectroscopy enables the in situ analysis of sample composition, aiding in the understanding of the geological history of extraterrestrial bodies. To complement spectroscopic data, the inclusion of 3D imaging is proposed to provide unprecedented contextual information. The morphological information aids material characterization and hence the constraining of rock and mineral histories. Assigning height information to lateral pixels creates topographies, which offer a more complete spatial dataset than contextual 2D imaging. To aid the integration of 3D measurement into future proposals for rover-based laser spectrometers, the relevant scientific, rover, and sample constraints are outlined. The candidate 3D technologies are discussed, and estimates of performance, weight, and power consumptions guide the down-selection process in three application examples. Technology choice is discussed from different perspectives. Inline microscopic fringe-projection profilometry, incoherent digital holography, and multiwavelength digital holography are found to be promising candidates for further development.

Online nonlinearity elimination for fringe projection profilometry using slope intensity coding

Abstract The nonlinearity effect in the system of fringe projection profilometry (FPP) will cause the non-sinusoidal deviation of the fringe patterns, inducing ripple-like phase errors and further affecting the measurement accuracy. This paper presents an online nonlinearity elimination method based on slope intensity coding. Two sequences of sinusoidal phase-shifting fringe patterns with different frequencies, and one slope intensity pattern with one uniform intensity pattern are projected. The equations of the nonlinearity response are established using the defined mean and modulation parameters, the captured uniform intensity and two extracted background intensities. The nonlinearity response coefficients determined by solving the equations are used for pixel-wise nonlinearity correction on the captured images, which are employed for computing the wrapped phase, and further obtaining continuous phase by the multi-frequency phase unwrapping method. Experimental results demonstrated that the proposed method could eliminate the nonlinearity-induced phase error online by using fewer images, and maintain the reliability of phase unwrapping in the measurement of isolated objects with complex surface.

Three 1-bit Speckle-embedded pulse-width modulation patterns for robust absolute 3D measurement

Abstract In 3D shape measurement techniques using structured light, 1-bit pulse-width modulation (PWM) patterns and 1-bit speckle patterns can be projected at high speed. However, when combining PWM and speckle patterns to integrate their advantages, the decoupling problem is insurmountable. In this work, a novel 1-bit speckle-embedded pulse-width-modulation (SP-PWM) patterns was proposed to achieve absolute 3D shape measurement using only three binary patterns. Our method consists of three main steps: First, a sinusoidal pattern reconstruction network (SPRNet) was proposed to eliminate the high-order harmonics and speckle patterns in the SP-PWM patterns and obtain high-quality sinusoidal patterns. Second, a multi-temporal spatial correlation matching algorithm (MTSCMA) was proposed to obtain a coarse disparity map from the three SP-PWM patterns. Third, the high-accuracy wrapped phase map is used as an additional constraint for refining the coarse disparity map to obtain the final high-accuracy disparity map for absolute 3D measurement without phase unwrapping. Our method combines the advantages of fringe projection profilometry techniques for high-precision wrapped phase retrieval and speckle correlation matching algorithms for robust and unambiguous disparity map calculation. The experimental results demonstrated that our method could realize high-precision absolute 3D shape measurement with an accuracy of 0.057mm using only three 1-bit SP-PWM patterns. Furthermore, different simulation noises were used to demonstrate the robustness of the proposed method.

Multi-view fringe projection profilometry based on phase texture and U-Net

Multi-view fringe projection profilometry based on phase texture and U-Net

Parameter selection on a multi-exposure fusion method for measuring surfaces with varying reflectivity in microscope fringe projection profilometry

As industrial and scientific advancements continue, the demand for precise measurement of three-dimensional (3D) shapes and surfaces is steadily increasing. However, accurate 3D measurement of certain surfaces, especially those with varying reflectivities, has always been a challenging issue. Multi-exposure fusion methods have shown stable, high-quality measurement results, but the selection of parameters for these methods has largely been based on experience. To address this issue, this paper has improved the multi-exposure fusion method and introduced a guided approach for parameter selection, significantly enhancing the completeness of measurement results. Additionally, a comparative model is developed to experimentally validate the specific impacts of Gaussian window variance, optimal grayscale range, and attenuation factor variance on the integrity of 3D reconstruction. The experimental results demonstrate that under the guidance of the parameter adjustment method proposed in this paper, the multi-exposure fusion for measuring the 3D topography of high-dynamic surfaces improves the restoration coverage from the original 86% (bright areas) and 50% (dark areas) to over 99%. This provides a selection strategy for parameter adjustment guidance in precise measurements based on the multi-exposure method.

High-Efficiency Dynamic Three-Dimensional Topography Measurement Using the Phase Shift Generation Method

In the fringe projection profilometry (FPP), the traditional phase-shifting (TPS) algorithm and the Fourier transform (FT) algorithm are beset with a conundrum where measurement efficiency and conflicts with measurement accuracy, thereby limiting their application in dynamic three-dimensional (3D) measurements. Here, we propose a phase shift generation (PSG) method, which acquires the sinusoidal fringes by sparse sampling and reconstructs the complete phase-shifting sequence by generating the missing fringes with superimposed coupling of adjacent fringes. According to our proposed PSG method in which the sinusoidal fringe sequence size is about half of the TPS method, meaning that the PSG method will be timesaving in the phase-shifting sequence sampling process. Moreover, because of the utilization of multiframe fringes, our PSG method allows for a more accurate measurement than the FT method. Both simulation and experimental results demonstrate that our proposed PSG method can well balance the measurement accuracy and efficiency with a lower sampling rate, bearing a great potential to be applied in both scientific and industrial areas.

Adaptive double PCA method of phase extraction in fringe projection profilometry

Adaptive double PCA method of phase extraction in fringe projection profilometry

A ternary complementary Gray code phase unwrapping method in fringe projection profilometry

This paper proposes a phase unwrapping method based on ternary Gray code in fringe projection profilometry. This method does not require additional projection images or any prior knowledge. It accurately determines the segmentation thresholds of the ternary Gray code using only the average intensity and intensity modulation calculated from phase-shifting fringes, effectively reducing projection costs and achieving lower codeword errors. Additionally, a segment selection strategy suitable for the ternary Gray code is proposed. By employing the complementary method, three sets of candidate orders with certain codeword shifts can be generated. The wrapped phase is then divided into three segments: upper, middle, and lower segments, and correspondingly matched with the central regions of the three sets of candidate orders, thus enabling the solution of the absolute phase. Through the segment selection strategy, error-prone areas at the edges of orders are effectively avoided, further enhancing the accuracy of measurement results. Compared to the binary Gray code, the ternary complementary Gray code can encode up to 3m−1 orders when the number of Gray code patterns is m; when the required number of encoding orders is K, only ⌈log3K⌉+1 Gray code patterns are needed. Its additional value capacity precisely enables it to encode more orders, thereby increasing the frequency of phase-shifting fringes and further enhancing measurement efficiency. A series of experiments validates the feasibility, robustness, and superiority of this method.

Deep learning-based frequency-multiplexing composite-fringe projection profilometry technique for one-shot 3D shape measurement

The pursuit of precise and efficient 3D shape measurement has long been a focal point within the fringe projection profilometry (FPP) domain. However, achieving precise 3D reconstruction for isolated objects from a single fringe image remains a challenging task in the field. In this paper, a deep learning-based frequency-multiplexing (FM) composite-fringe projection profilometry (DFCP) is proposed, where an end-to-end absolute phase retrieval network (APR-Net) is trained to directly recover the absolute phase map from a FM composite fringe pattern. The obtained absolute phase map exhibits exceptional precision and is devoid of spectrum crosstalk or leakage disturbance typically encountered in traditional FM techniques. APR-Net is intricately crafted, incorporating a nested strategy along with the concept of centralized information interaction. A diverse dataset encompassing various scenarios and free from spectrum aliasing is assembled to guide the training process. A seven-map loss calculation scheme is employed to guide the training process, of which the efficacy is proved through ablation experiments. In the first qualitative experiment, DFCP demonstrates comparable phase accuracy to ground truths with 47 fewer projected images, outperforming other three methods with mean absolute phase errors of 0.0052 rad, 0.1761 rad, 0.0169 rad, and 0.0139 rad. The second qualitative experiment and the quantitative evaluation respectively prove DFCP’s capability in high dynamic range 3D measurement and in precise 3D measurement, with sphere diameter errors of 0.0780 mm and 0.0726 mm, and a spherical centroid distance error of 0.0555 mm.

High dynamic range 3D shape reconstruction: Combination of adaptive fringe intensity, iterative exposure time adjustment and least quadratic curve fitting

For three-dimensional (3D) reconstruction of specular surfaces based on fringe projection profilometry (FPP), a hybrid approach combining adaptive fringe projection, iterative exposure time adjustment and least quadratic curve fitting is proposed. First, we propose an adaptive projection fringe intensity generation method in the camera image. Second, we build a new pixel matching between the camera and projector, and correct the pixel matching on the specular surfaces based on linear interpolation. Third, we propose an iterative exposure time adjustment method based on saturated pixel statistics. Fourth, we fuse the multiple results to obtain the final 3D reconstructed surface. Finally, we adopt a least quadratic curve fitting algorithm to fit the residual phase errors. Experiments confirmed the validity of the proposed method.

Phase Error Reduction for a Structured-Light 3D System Based on a Texture-Modulated Reprojection Method.

Fringe projection profilometry (FPP), with benefits such as high precision and a large depth of field, is a popular 3D optical measurement method widely used in precision reconstruction scenarios. However, the pixel brightness at reflective edges does not satisfy the conditions of the ideal pixel-wise phase-shifting model due to the influence of scene texture and system defocus, resulting in severe phase errors. To address this problem, we theoretically analyze the non-pixel-wise phase propagation model for texture edges and propose a reprojection strategy based on scene texture modulation. The strategy first obtains the reprojection weight mask by projecting typical FPP patterns and calculating the scene texture reflection ratio, then reprojects stripe patterns modulated by the weight mask to eliminate texture edge effects, and finally fuses coarse and refined phase maps to generate an accurate phase map. We validated the proposed method on various texture scenes, including a smooth plane, depth surface, and curved surface. Experimental results show that the root mean square error (RMSE) of the phase at the texture edge decreased by 53.32%, proving the effectiveness of the reprojection strategy in eliminating depth errors at texture edges.

Dual-frequency phase unwrapping based on deep learning driven by simulation dataset

In fringe projection profilometry (FPP), hierarchical temporal phase unwrapping is used to reliably eliminate phase ambiguity in complex scenarios, in which triple-frequency hierarchical temporal phase unwrapping (THPU) has excellent accuracy but lower measurement speed, while dual-frequency hierarchical temporal phase unwrapping (DHPU) requires only two wrapped phases, but it is fragile to noise contamination. Some researchers have shown that deep learning techniques can be used to overcome this dilemma, but the high dataset building cost makes it difficult to apply them rapidly. In this paper, a novel simulation dataset generation method for FPP is proposed, which requires only a set of pre-collected fringe sequences on the reference plane along with some historical masks to generate a dataset, which is then used to train a neural network. The appropriately trained neural network is able to perform dual-frequency phase unwrapping in actual scenarios based on the principle of regression and segmentation respectively, and both of them show accuracy comparable to THPU, but segmentation is slightly better than regression. We verify its effectiveness through comparative experiments and demonstrate its robustness with different degrees of noise contamination. We believe that this paper can provide potential and beneficial ideas for simulation dataset driven deep learning techniques for optimizing various processes in the field of optical measurements.

Absolute phase retrieval based on spatial ternary phase coding with circular fringe projection

Fringe projection profilometry (FPP) based on temporal phase unwrapping (TPU) plays a vital role due to its high precision and fast speed in three-dimensional (3D) measurements. However, a fundamental contradiction arises between the necessity for numerous auxiliary patterns and the goal of enhancing measurement efficiency, particularly when ensuring a big codeword space for fringe order. Thus, a spatial ternary phase coding (STPC) method is proposed, which achieves a big codeword space by projecting just one auxiliary coding pattern while ensuring a robust decoding. By equally dividing each fringe period into several segments, ternary coding of fringe order is accomplished using an arccosine operation to code three distinct phase values within each segment. Meanwhile, circular grating is adopted due to its excellent performance in phase sensitivity, which offers a higher measuring accuracy in 3D reconstruction. Accordingly, a robust TPU is achieved by combining the introduced phase unwrapping scheme and the circle center-cosine theorem (CCCT) strategy. Furthermore, high-efficiency measurements are attained through projecting three phase-shifting circular gratings and one STPC grating, rendering it suitable for real-time measurements. Experimental results with various static objects and real-time scene demonstrate the feasibility and validity of the proposed method.

Systematic Radio Telescope Alignment Using Portable Fringe Projection Profilometry

In 2019, the Event Horizon Telescope (EHT) released the first-ever image of a black hole event horizon. Astronomers are now aiming for higher angular resolutions of distant targets, like black holes, to understand more about the fundamental laws of gravity that govern our universe. To achieve this higher resolution and increased sensitivity, larger radio telescopes are needed to operate at higher frequencies and in larger quantities. Projects like the next-generation Very Large Array (ngVLA) and the Square-Kilometer Array (SKA) require building hundreds of telescopes with diameters greater than 10 ms over the next decade. This has a twofold effect. Radio telescope surfaces need to be more accurate to operate at higher frequencies, and the logistics involved in maintaining a radio telescope need to be simplified to support them properly in large quantities. Both of these problems can be solved with improved methods for surface metrology that are faster and more accurate with a higher resolution. This leads to faster and more accurate panel alignment and, therefore, a more productive observatory. In this paper, we present the use of binocular fringe projection profilometry as a solution to this problem and demonstrate it by aligning two panels on a 3-m radio telescope dish. The measurement takes only 10 min and directly delivers feedback on the tip, tilt, and piston of each panel to create the ideal reflector shape.

Fringe Projection Profilometry for Three-Dimensional Measurement of Aerospace Blades

The aero-engine serves as the “heart” of an aircraft and is a primary factor determining the aircraft’s performance. Among the crucial components in the core of aero-engines, aero-engine compressor blades stand out as extremely important. They are not only numerous but also characterized by a multitude of parameters, making them the most complex parts in an aero-engine. This paper aims to address the trade-off between accuracy and efficiency in the existing measurement methods for asymmetric blades. Non-contact measurements were conducted using a structured light system composed of a stereo camera and a DLC projector. The point cloud data of the blades are processed using methods such as the PCA (Principal Component Analysis) algorithm, binary search, and least squares fitting. This paper established a fringe-projection profilometry light sensor system for the multi-view measurement of the blades. High-precision rotary tables are utilized to rotate and extract complete spatial point cloud data of aviation blades. Finally, measurements and comparative experiments on the blade body are conducted. The obtained blade point cloud data undergo sorting and denoising processes, resulting in improved measurement accuracy. The measurement error of the blade chord length is 0.001%, the measurement error of blade maximum thickness is 0.895%, compared to CMM (Coordinate Measuring Machine), where the measurement error of chord is 0.06%.

Comparative analysis of circular and linear fringe projection profilometry: from calibration to 3D reconstruction

This study compares the accuracy of circular and linear fringe projection profilometry in the aspects of system calibration and 3D reconstruction. We introduce, what we believe to be, a novel calibration method and 3D reconstruction technique using circular and radial fringe patterns. Our approach is compared with the traditional linear phase-shifting method through several 2 × 2 experimental setups. Results indicate that our 3D reconstruction method surpasses the linear phase-shifting approach in performance, although calibration efficiency does not present a superior performance. Further analysis reveals that sensitivity and estimated phase error contribute to the relative underperformance in calibration. This paper offers insights into the potentials and limitations of circular fringe projection profilometry.

Weakly Supervised Depth Estimation for 3D Imaging with Single Camera Fringe Projection Profilometry.

Fringe projection profilometry (FPP) is widely used for high-accuracy 3D imaging. However, employing multiple sets of fringe patterns ensures 3D reconstruction accuracy while inevitably constraining the measurement speed. Conventional dual-frequency FPP reduces the number of fringe patterns for one reconstruction to six or fewer, but the highest period-number of fringe patterns generally is limited because of phase errors. Deep learning makes depth estimation from fringe images possible. Inspired by unsupervised monocular depth estimation, this paper proposes a novel, weakly supervised method of depth estimation for single-camera FPP. The trained network can estimate the depth from three frames of 64-period fringe images. The proposed method is more efficient in terms of fringe pattern efficiency by at least 50% compared to conventional FPP. The experimental results show that the method achieves competitive accuracy compared to the supervised method and is significantly superior to the conventional dual-frequency methods.

Large depth-of-field fringe projection profilometry applied to defocused scenes

Large depth-of-field fringe projection profilometry applied to defocused scenes

Improved two-frequency temporal phase unwrapping method in fringe projection profilometry

Improved two-frequency temporal phase unwrapping method in fringe projection profilometry